Embedded audio/video playout program frequently requests keyboard focus

ABSTRACT

Computer software allowing enhanced control of the playout of audio/video works on a computer system. In various embodiments, the software allows key events from dedicated audio/video keys, whether part of a full sized keyboard or on a hand held remote, to control the actions of an audio/video playout program without requiring the user to direct the key event focus of the operating system to the audio/video playout program. Also, the invention distinguishes between key presses from a local, full sized keyboard and key presses from a remote keyboard so that the audio/video playout program can enlarge its screen display when a key event is received from the remote keyboard. In one embodiment, the invention constantly instructs the operating system to move the focus to the audio/video playout program. In another embodiment, if the focus is received by any of various windows in a display, software associated with the window forwards to the audio/video playout program any key events received from audio/video keys. In a third embodiment, audio/video key event data is routed to the audio/video playout program by a method that does not use the key event features of the operating system, such as by using a key board wedge server program to serve key events to audio/video client programs. In a fourth embodiment, the operating system is modified so that it has two separate focuses, one for a text keyboard and a second focus for audio/video keys.

This application is a continuation of U.S. Ser. No. 13/854,895 andclaims priority from Ser. No. 09/716,052 filed Nov. 17, 2000, issued asU.S. Pat. No. 6,976,216.

FIELD OF INVENTION

This invention relates to use of dedicated audio/video control keys oncomputing devices that execute programs for playing audio or videoworks.

BACKGROUND

Since compact disc drives were added to personal computers many yearsago, personal computers have been used for playing first audio and thenvideo works of authorship. The common implementations use either a mouseor a standard text keyboard to control the playout software. If a mouseis used, images of buttons for some of the customary playout controlfunctions (play, stop, pause, fast forward, rewind, skip to next or lasttrack, volume, mute, and change channel) are displayed on the screen.The mouse controls cursor movement, and a click of the mouse when thecursor is over a button causes the playout software to control theplayout accordingly. If a standard text keyboard is used, keys of thekeyboard will be selected for each of the control functions listedabove, such as “P” for play and right and left arrows for fast-forwardand rewind. The audio/video playout control software is typically loadedto the computing device from a transportable disc or may be downloadedacross a network for installation or may be embedded in a markuplanguage page such as with use of a Java applet or ActiveX controlembedded in an HTML page which is transparent to the user and requireslittle to no action by the user to install or begin execution of thesoftware.

Users of computer systems to play audio/video works can add remotecontrol devices such as those commonly used for televisions or for audiostereo systems to allow control of the playout of audio/video works fromanywhere in the same room. Radio or infrared receiving devices for suchremote controls can be added to an input port on the computer, such as aUSB (Universal Serial Bus) port. Then, a specialized audio/video workplayout program is added which includes instructions to receivekeystroke data through the port and interpret said data in order torecognize remote control key presses. This allows the remote key pressesto effect audio/video playout according to which key is pressed.

As audio/video playout programs became commonplace on personalcomputers, it became desirable to be able to control a wide variety ofaudio/video control programs that are designed for text keyboard inputusing a hand-held remote. To accomplish this, special software(sometimes called a “wedge”) can be installed on the computer to controlcommunications through the port, receive keystroke data from the remotecontrol, translate the keystroke data to standard text keystroke datasuch as “P” for play, and provide the standard keystroke data to theoperating system which provides it to the playout control software.Thus, with the remote control added to the computer system and thespecial control software installed, the remote control can simulatekeystrokes at the text keyboard to control audio/video playout controlsoftware designed for receiving text keyboard input. Then the user cancontrol the audio/video playout control software using either the textkeyboard or the remote control.

There are two serious deficiencies with this system. First, the systemcan no longer use for their original intended purpose the standard textkeys that are used to indicate audio/video playout control. For example,a playout control program that uses “P” to indicate “pause playout” orleft and right arrows to indicate “previous audio/video work” or “nextaudio/video work” are not able to use these same keys to allow the userto type a music artist or song name into an edit box. Second, thetranslator program must be tailored to work with differing varieties ofaudio/video playout programs, all of which use differing keys torepresent each playout control action.

Newer operating systems, such as Microsoft's Windows 2000, address theseproblems by creating a standard set of key codes that are passed toapplications when a playout control key is pressed. These standard keycodes are used specifically for keyboards or other input devices withdedicated keys for playout control purposes. This method carries theadvantage that any application can be easily made to recognize playoutcontrol key presses.

This approach still suffers from deficiencies because the operatingsystem delivers each playout control key press event to softwareprograms in the same way that a text keyboard key event would bedelivered. Specifically, if a user presses a key when several softwareprograms are running, the operating system must determine which one ofthose applications is to receive the notification of key-press, or “keyevent”. The existing approach is to send key press information only tothe application window with which the user is actively interacting. Thisapproach makes sense for most applications, but audio/video playoutcontrol applications—in particular, audio player programs—are anexception.

It is a common occurrence to simultaneously execute an audio/videoplayout program and another program that demands user input, such as aword processor. Typically the audio/video playout program is run in thebackground because it requires less user attention to operate. However,because existing systems send keystrokes to the foreground program, theuser must explicitly select the audio/video program to be the foregroundprogram (usually by moving the mouse over the audio/video playoutprogram window and clicking the mouse), before being able to sendaudio/video playout key-press events to the audio/video playout program.This adds an additional, inconvenient and unnecessary step toaudio/video playout program operation.

The requirement that the user must specifically select the audio/videoplayout control application to perform audio/video playout control isparticularly confusing to the user when presented with a web page thatconsists of a markup language with one or more embedded applications. Inthis case, though the user is presented with what appears to be a singleweb application, it is in reality a conglomerate of one or moreapplications as well as text, graphics, and user interface items such ascheckboxes or text boxes where one may provide input using the keyboardor mouse. On the web page, in order to control the audio/video playoutapplication, the user must select the portion of the web page thatdisplays the playout application to place it in the “foreground” beforebeing able to press a playout control key that will control the embeddedaudio/video playout program. Furthermore, if the user were to click onany other web page constituents, the playout application would no longerreceive keyboard events, preventing the user from using the dedicatedaudio/video keys to control audio/video playout.

In modern operating systems, where multiple computer programs mayoperate concurrently as one or more “windows” on a screen, or as one ormore “child” windows of a web browser, standard keyboard-press eventsare communicated by the operating system to the window that has the“keyboard focus” (is in the “foreground”). The keyboard focus istypically assigned by the user to a window by placing the mouse cursorover a window and clicking a mouse button. If the assigned window is awindow where the user can type text, often a flashing text cursor can beseen in the window that has the keyboard focus. Windows that do not haveflashing text cursors typically indicate visually that they have thekeyboard focus by changing the color of the application's primarywindow's title bar.

On existing computers equipped with a common (text) keyboard as well asdedicated keys for audio/video playout control (sometimes on a wirelesshand-held remote), such dedicated audio/video control key-presses arecommunicated to the application window that has the keyboard focus, justas are standard key presses. Alternatively, the operating softwaredelivers audio/video key events only to a specialized set of programscustomized for the device.

SUMMARY

This invention comprises systems that enable a software program toreceive audio/video key events without requiring the user to explicitlydirect the keyboard focus to the playout program. The inventionsimplifies the user interface because it allows audio/video playoutbuttons to control a web-embedded audio/video player even if the userhas not explicitly set the keyboard focus within the web page to thataudio/video player.

Note that though the figures in this document indicate that theaudio/video playout window is visible to the user at all times, it is acommon technique to hide a program window by specifying dimensions ofzero width and zero height for the program's window. In this case, theinventions as described are still effective, and indeed more importantbecause the user has no clear way to set the keyboard focus to theapplication by using the mouse or keyboard.

The invention includes methods that allow a computer user to communicateaudio/video key events to applications that do not have a window in theforeground. This is accomplished by creating two keyboard focuses: onefor regular text input and a second for audio/video keyboard input. Byhaving two separate keyboard focuses, the user may assign theaudio/video keyboard focus to any playout program that may be running inthe foreground or background on the computer. The user may then set thetext keyboard focus to any other computer program, without affecting theplacement of the audio/video keyboard focus. Furthermore, the user mayreassign the audio/video keyboard focus to another executing playoutprogram at any time. Additionally, the user may enter a default mode,whereby the audio/video focus window is the same as the text keyboardfocus window. This frees the user from thinking about multiple keyboardfocuses when their usage is not required.

Another shortcoming of existing systems that is addressed by thisinvention is the need for playout programs to be able to determinewhether the user has pressed a playout control key on a keyboard that istypically used at short range (such as a typing keyboard) or on akeyboard that is typically used at long range (such as a hand-heldremote). This is desirable because a program may be better suited todifferent modes of operation when the user presses audio/video controlbuttons on a hand-held remote. For example, it is desirable totemporarily display text in an enlarged font if the user presses buttonson a hand-held remote, because presumably the user is operating thecomputer from a distance that makes regular screen text viewingdifficult. This is accomplished by delivering, along with key eventnotification data, an indication of whether the key was pressed on ashort-range keyboard or a long-range keyboard. Alternatively, a new setof distinct codes may be used to identify each long-range keyboard key.

The invention is most useful on computers equipped with specializedkeyboards that have audio/video playout keys or on computers that havean auxiliary audio/video keyboard (“handheld remote”) to controlaudio/video playout programs. However, many of the embodiments describedare useful on computer systems with playout programs that use regulartext keys (“P” for Play, “S” for Stop, “Up Arrow” for Volume Up) thatare dedicated to controlling audio/video playout.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of a web browser displaying typical web pageconstituents.

FIG. 2 is a top view of a hand-held keypad whose keys consist only ofaudio/video playout control keys.

FIG. 3 is a top view of a traditional keyboard with the addition ofspecialized audio/video playout control keys.

FIG. 4 is a diagram of program window hierarchy on a modern operatingsystem “desktop” display.

FIG. 5. is a diagram of the communications between a “wedge server”program that is added to an operating system to enable an auxiliaryaudio/video keyboard focus, and the “event client” programs that receiveaudio/video key events from the wedge program by the Microsoft COMprotocol.

DETAILED DESCRIPTION

It is common to create a web page that is composed by a markup languageand/or scripting language that specifies the arrangement of web page“constituents”: text, images, hyperlinks, keyboard or mouse input boxes,audio/video works, and embedded applications such as Java applets orActiveX controls. FIG. 1 shows a block diagram of an example web page 13as displayed in a web browser display window 10.

For the purposes of the present invention, a web browser is any computerapplication that interprets a data set in a generalized form suitablefor interpretation by many different computer systems to create adisplay to a user, and a web page is such a data set in any generalizedform, including a frame. A web browser is typically designed tointerpret human-readable markup languages that describe the arrangementof web page constituents and present said arrangement to the computeruser. Such markup language web pages are often sourced from a servercomputer over a computer network such as the Internet, using HTTPprotocol, but other transmission protocols may be used, and web pagesmay originate from other places such as a storage device in the computeron which the web browser is executing.

The markup language described is preferably HTML but it can be anysequence of human readable computer instructions that are interpreted bya web browser to specify the arrangement of both useful information orentertainment and embedded computer programs (Java applets/ActiveXcontrols). Such currently known markup languages include SGML (StandardGeneralized Markup Language), WML (Wireless Markup Language), HDML(Handheld Device Markup Language), and Extensible Markup Language (XML).Other similar languages, some of which are not called “markup”languages, are in development and still more will be developed in thefuture. For purposes of the present invention, the essential function ofthe markup language is that it allows an executable program, whethercompiled or interpreted or semi-compiled to non human-readable byte-codewhich is then interpreted, to be loaded by the web browser as aconstituent of a web page, and to automatically execute on the devicewithout requiring attention or action by the user while the user isreceiving information desired by the user from the web browser. Theinformation typically includes visually displayed information includingtext, but that is not an essential component for purposes of thisinvention. The information could be entirely auditory, such as music orspoken words.

Markup languages are often accompanied by scripting languages such asJavaScript, JScript, VBScript, or ECMAScript that are human-readableinstructions interpreted by the web browser/operating system in order toaffect how each web page constituent appears and behaves as a functionof time and as a function of user input such as keyboard and mousemovement or clicks. Such scripting languages are often used tocoordinate the visual display or operation of the constituents of a webpage. This is preferably done using the DOM (Document Object Model)whose essential function is to provide a standard definition of therelationships of various web page constituents and to provide anomenclature that scripting languages may use to access them. Othersimilar languages, some of which are not called “scripting” languages,are in development and still more will be developed in the future.

For the purposes of the invention, the essential function of the scriptprogram is to cause programs on the client machine to detect and respondto keystrokes or mouse clicks that are sent to the web browser, webpage, or web page constituents, and to operate on web page constituentsin order to change their appearance, keyboard focus, or if they areembedded programs, to communicate messages to the programs.

It is common to imbed into a web page an audio/video playout programsuch as the Microsoft Media Player ActiveX control or RealPlayer ActiveXcontrol. This allows the user to hear or view an audio/video work withinthe context of the web page without opening a separate applicationwindow for the audio/video player program.

Sometimes the player program will allow the user to control playout bydedicating certain keys as playout control keys such as pressing theletter “P” to pause or pressing the up or down arrows to increase ordecrease volume. Traditional keyboard keys are usually used for thesefunctions because most computers and operating systems do not havespecialized keys and corresponding key codes for audio/video playoutcontrol. However, it is expected that playout programs will be designedto respond to such specialized keys as they become commonplace onkeyboards, as in FIG. 3, or as auxiliary hand-held keypads, as in FIG.2.

On computers and operating systems that do not have specializedaudio/video control keys, it is common to add a specialized wirelesskeypad to a computer, and add a specialized program to receive data fromsaid wireless keypad and translate it into the traditional keystrokeswhich are sent to an audio/video playout application in order to effectthe desired playout control.

In the preferred embodiment of this invention, the keypad is an infraredremote such as those used to control traditional home entertainmentelectronics, and the infrared signals are translated to digital codesthat are sent to a personal computer through the USB port. However, theremote may alternatively use radio frequency communication to transmitkey press information, and the means of communication to a PC may be theserial port, IEEE 1394 port, IrDA port, ISA (Industry StandardArchitecture) bus, PCI (Peripheral Component Interconnect) bus,BlueTooth, 802.11, or any other communications port or bus, includingthose that are still under development or may be developed in thefuture.

Regardless of whether the audio/video playout program detectstraditional key events, or specialized audio/video key events, existingoperating systems deliver both types of key events to the playoutprogram in the same manner. That is, the window that has the keyboardfocus will be the sole window to receive key events.

Optionally, the application window with keyboard focus may choose topass key press events to its “parent” window—the window in which theapplication window resides. Common operating systems adopt the notion ofchild windows and parent windows, as shown in FIG. 4. The topmost windowis typically called the “Desktop” window 19 and application displays 40,44 have primary windows 41, 45 that are the child windows of the desktopwindow 19. In FIG. 4, the desktop window 19 is the parent to two childapplication windows 41, 45. Furthermore, any child windows, such as 41and 45 may act as the parent windows to their own child windows. Windows42 and 43 are child windows to the parent window 41. Windows 46 and 47are child windows to the parent window 44. In this document, a parentwindow and its child windows (and their child windows, etcetera) will bereferred to collectively as a “window family” designated by the name ofthe parent.

An essential function of this window hierarchy is to provide a clearsequence that describes how user input events, such as key presses ormouse clicks, are processed by applications. When a child window has thekeyboard focus, it is the first window to receive key events from theoperating system. This window then has the option to pass the key eventto its parent window. The parent window may then act on the key event,sometimes by passing the event to its parent.

In the preferred embodiment, the operating system is Microsoft Windows2000, but the invention applies to all operating systems that implementthe same parent-child window hierarchy and share this system of keyevent propagation.

Most audio/video playout control programs exist as stand-aloneapplications. That is, they have their own primary application window41, 45. This means that even if the application has a child window thathas the keyboard focus, the parent window will receive key pressnotification from the child window as long each as child window passeskey press events up to its parent window. To allow dedicated audio/videoplayout keys to function at all times, existing playout programs consistof only a primary application window or they ensure that everyuser-selectable child window passes playout control key events to itsparent. Then the primary application window, when receiving key events,will always affect audio/video playout. In this way, as long as one ofthe application's windows has the keyboard focus, the desired playoutcontrol will occur as a result of playout key presses.

Web Page Embedded Playout Programs

This technique becomes less effective when the audio/video playoutprogram is not a stand-alone application, but instead is embedded in aweb page. As shown in FIG. 1, a web page embedded program window 50 is achild window of the web page window 13. This means that, unless thewindow of the web page embedded program has the keyboard focus, or oneof its child windows has the keyboard focus, the embedded program willnot receive keyboard events. For example, in the preferred embodiment,the web browser is Microsoft's Internet Explorer 5.5. In this webbrowser, when a web page is first opened, the web page window 13automatically receives the keyboard focus. Since the embeddedaudio/video playout program window 50 is a child window of the web pagewindow 13, it will not receive any key press events. This can beconfusing to a user if the web page presents an audio/video work as thecentral constituent of the web page. Even for users who understand thatthe embedded programs must have the keyboard focus to receive key pressevents, it is inconvenient for the user to have to select theaudio/video application in order to control playout using key presses.Web pages often consist of text, graphics, or hyperlinks that the usermay want to click, for example to select text for copying, or to followa hyperlink. This clicking action will cause the web page window 13 toreceive the keyboard focus, and keyboard key events will no longer bedelivered to the embedded application. This is particularly inconvenientfor the users who may inadvertently set the keyboard focus to a windowthat is not the audio/video player application, and then step away fromthe computer and attempt to perform playout control using a hand-heldremote keypad. Beyond the inconvenience, it may in fact be impossible orextremely difficult for the user to set the keyboard focus to the playerprogram window 50 due to the fact that some web pages may choose to hidethe player program window 50 from the user by setting its height andwidth dimensions to zero or some small number.

In order to solve this problem, two solutions are presented.

1. Embedded Program or Associated Script Frequently Requests KeyboardFocus

In a first embodiment of the invention, as shown in FIG. 1, theweb-embedded audio/video application is the only web page constituentthat needs to process and respond to key events. In this case, theembedded player includes instructions to continuously request theykeyboard focus from the operating system/web browser. The rate ispreferably slow enough to ensure minimized processor usage, but highenough to ensure lively user interface responsiveness, such as between100 Hz and 1 Hz in an Intel Pentium processor running at 300 MHz. Inthis manner, even if the user clicks on a part of the desktop window 19or a part of the web page window 13 outside of the audio/video programwindow 50, the keyboard focus will quickly be redirected to the properprogram window 50. There may, in fact be other embedded programs on thisweb page which are capable of receiving the keyboard focus byuser-specification. However, as long as those programs do not need toreact to keyboard input, it is not a problem for the user if thekeyboard focus is immediately reassigned to the audio/video playoutprogram window.

While some operating systems and web browsers, such as Windows 2000 andInternet Explorer 5.5, allow the embedded application itself to obtainthe keyboard focus by request, it is also possible to use a scriptinglanguage to cause the web browser to set the keyboard focus to aspecific web page constituent. Thus, in a preferred embodiment, theJavaScript language instructs the web browser to set the keyboard focusto the embedded audio/video program window 50 on a continual basis.

Note that in this solution, the embedded application does not includeinstructions that may affect its keyboard focus relative to other webpage constituents. Instead, the web browser, by interpreting the webpage markup and scripting languages, will set the keyboard focus to theappropriate web page constituent. This is preferred because keyboardfocus instructions are not included in the embedded program. Thisenables the same program to be used in a wide variety of web pages,allowing the web page composer freedom to determine with a script howand if the embedded audio/video application should have the keyboardfocus. In this manner, the same audio/video application can be utilizedon a wide number of web pages under different usage scenarios.

2. Browser and all Constituents Instruct Audio/Video Control Program

The above solution has a drawback when there are other web pageconstituents besides a playout control program to which the user maywant to intentionally assign the keyboard focus. For example, in FIG. 1,a web page may incorporate a search function where the user may type into a key entry window 101 that is a child window of the web page window13 rather than the window of the playout control program. In thisarrangement, it would be undesirable to continuously assign the keyboardfocus to the playout control program window 50 because the user may wantto place the keyboard focus in the text box window 101 in order to typein, for example, the title of an audio/video work that the user wouldlike to see or hear. In this case, a better approach is to writescripting language instructions that instruct the browser to use awindow within its window family to intercept and interpret key eventsintended for web page constituents that are not part of the embeddedaudio/video player program.

If the web browser were capable of redirecting key events that aredirected to any child window of the browser window to an embeddedprogram, it would be sufficient to use a script to cause the browser tosimply capture all key events to all web page constituents, and redirectkey events resulting from audio/video control key events to theaudio/video player window 50. However, for security reasons, known webbrowsers do not allow scripting programs or embedded applications tocommunicate key or mouse events to other web page constituents (which ishow a redirection of a key event would be done).

The solution of this invention is to place within the embeddedaudio/video control program a means for the scripting language tocontrol playout by accessing “methods” of the audio/video program whichare made available to the scripting language by way of the DocumentObject Model specification. The preferred implementation uses theJavaScript language to cause the browser to use one of the browserfamily windows to capture all key events, then detect whether each keyevent is from an audio/video control key and, in the event that it is,to activate the corresponding audio/video control action by calling theavailable methods of the web-embedded application.

However, although contemporary browsers are able to intercept key eventssent to regular web page constituents such as text boxes, they areunable to intercept key events that are delivered directly by theoperating system to web embedded programs which may have their ownwindows within the browser family of windows and may request thekeyboard focus. In this case, each web embedded program on the web pagethat may have the keyboard focus must be written to detect audio/videocontrol key events and then notify the web-embedded audio/video playerto perform the corresponding playout control action via the DOM methoddescribed above.

Use of Uncommon Text Key Codes

An important extension of the invention for both of the aboveembodiments addresses the problem that most existing web browsers andoperating systems do not have specialized key codes for audio/videoplayout control keys. In this case, existing audio/video playoutprograms use traditional keys, such as “P” for play, “S” for stop, etc.to control playout. Another reason to use such keys is that browsers andscripting languages of present operating systems may not yet be capableof processing audio/video control keys.

However, using traditional keyboard buttons to control audio/videoplayout presents problems for the embodiments presented above. This isbecause these keys may be needed to indicate input into another web pageconstituent, such as a text box, for purposes other than playoutcontrol. If a playout control key is pressed while entering text data,playout control will inadvertently be activated.

An extension to the invention solves this problem by using key codesthat the browser and scripting language can process, but that would notnormally be used when typing into a text box. The following table is anexample of such key codes:

Playout Control Button Substitute Character Representation ASCII CodePLAY { 123 STOP } 125 PAUSE | 124 VOLUME UP {circumflex over ( )} 94VOLUME DOWN _(—) 95 REWIND < 60 FAST FORWARD > 62

Thus, a JavaScript program can intercept these keys when typed into atext box and perform the corresponding playout control action using theDocument Object Model to access the playout program. As described above,any other embedded program designed to receive text input should bewritten to receive the input from the browser in the form of a DOMcommand rather than key events from the operating system so that theseunusual characters will not appear as text in the program.Alternatively, the script program should cancel the key event, ifpossible, to ensure that it is not also interpreted as a key that isintended for the text box. Otherwise the character will appear in thebox and confuse the user.

Using such non-traditional key characters for playout control might atfirst be considered confusing to the user. However, the intention ofthis invention is for it to be used in conjunction with a specialized“wedge” program that interfaces with the audio/video playout controlkeyboard and translates each detected key-press to the key events in thetable above. Thus, the user does not need to remember these specializedkey codes. Instead the user simply presses a playout control key, whichis translated to the proper key code and delivered to the window thathas the keyboard focus.

A problem that arises here is that a web page composer may wish to embedan audio/video playout program that does not detect specialized keyssuch as those presented in the above table but nevertheless can obtainthe keyboard focus if selected by the user. Such as program would bedesigned to work with non-key event controls as described above. If sucha program were used with the above system and the user were toexplicitly set the keyboard focus to the audio/video playout program andthen press a playout control key, nothing would happen. The playoutprogram would not recognize these keys, and contemporary web page scriptprograms cannot intercept key events directed to embedded programs.

A solution to this problem is to write the web page script such that ifthe audio/video playout window receives the keyboard focus, the scriptwill immediately reassign the keyboard focus to the web page window 13so that the above described processes can take over.

Control of Audio/Video Keyboard Focus Whether Inside or Outside of a WebBrowser.

Unfortunately, none of the above embodiments solves the problem of auser wanting to enter text input in one program window, and sendaudio/video keyboard input to a different program window withoutchanging the text keyboard focus. For example (see FIG. 5), the user maywant to execute a word processing program 70 in the foreground, and atthe same time execute audio/video playout programs in background windows72, 73. A user might work at the computer using the word processorprogram, but occasionally step away from the computer and desire to usean audio/video control keypad to control a background audio/videoplayout program. Or one user may be word processing on the computerwhile a cohort controls a background audio/video playout program using ahand-held remote.

The following embodiments of the invention allow the audio/videokeyboard focus to be set to any number of computer program windows. Thisis accomplished by treating the audio/video keyboard focus as a separatekeyboard focus that is not set in the same way as the text keyboardfocus. In the following embodiments, any number of computer programs canreceive the audio/video keyboard focus without requiring reassignment ofthe text keyboard focus. Such programs may be embedded in a web page 73or they may execute as applications 72 that are independent of a webbrowser. In these embodiments, one program may have the audio/videokeyboard focus, while another program maintains the text keyboard focus.Thus, audio/video control key events are delivered to a singleaudio/video playout program 72 or 73, and text keystrokes are deliveredin the common way to the foreground window 70 that has the text keyboardfocus.

3. Multiple Keyboard Focuses Added to Existing Operating Systems

For security reasons, modern operating systems such as Windows 2000 donot allow programs to send key events to any window except theforeground window, nor do they allow programs to change the textkeyboard focus between executing programs. Additionally, operatingsystems that have key event codes for audio/video control keys deliversuch event codes to the window with the keyboard focus. Thus, tocommunicate audio/video control key events to a window that does nothave the keyboard focus, it is necessary to deliver audio/video keyevents by another means.

In a preferred embodiment, Microsoft's Component Object Model (COM) isused for auxiliary key event delivery. However, other inter-processcommunication methods, such as shared memory, pipes, sockets, CORBA,RMI, or Microsoft's general-purpose window message passing, may be usedinstead.

In existing computers with audio/video keyboards, the wedge programreceives information from the audio/video key hardware and translatesthat information into key events that are delivered to the operatingsystem, which in turn delivers the key events to the window that has thekeyboard focus. In this embodiment of the invention, as shown in FIG. 5,to deliver key events to various program windows, the wedge program 71uses Microsoft COM 74. First, the wedge program must know which programsare able to receive audio/video key events (herein referred to as “eventclient” programs 72, 73). Since there may be any number of suchprograms, at startup each event client uses the COM architecture tolocate and register its existence with the wedge “server” program 71. Inthis way, the wedge server 71 can keep track of all programs that arecapable of receiving audio/video key events. An important part of thisregistration process is that the client program 72, 73 provides thewedge server with information (sometimes called a “reference”, “handle”,or “pointer”) that allows the wedge server 71 to find and pass key eventinformation to the event client program 72, 73. By keeping a referenceto every program that needs to receive audio/video key events, the wedgeserver 71 can deliver key events to any one of these known key eventclients 72, 73. Similarly, an event client program 72, 73 notifies thewedge server when it would like to be removed from the list of eventclient programs. This is typically done when the event client programterminates execution. Alternatively, the wedge server may remove theevent client from its list upon detecting that the event client programis no longer executing.

When an audio/video key is pressed, only one of the event clientprograms is notified of the key event, and this program is considered tohave the “audio/video keyboard focus”. This simplifies the userinterface by ensuring that a single audio/video key-press controls onlyone audio/video playout program at a time. Accordingly, there must be ameans of specifying which program is to receive the audio/video keyboardfocus.

The default behavior is for the audio/video keyboard focus to beassigned to the same window that has the text keyboard focus. That is,when the user sets the text keyboard focus to a program window, thatprogram window also receives the audio/video keyboard focus. This is adesirable default behavior because it frees the user from having tothink about the assignment of the audio/video keyboard focus most of thetime. However, the user may override this default behavior in order toset the audio/video keyboard focus to a window other than the windowwith the text keyboard focus.

To implement the above method, the wedge server 71 in default mode sendsaudio/video key events to the operating system, which passes them to thewindow with the text keyboard focus. When the user selects a particularwindow to retain the audio/video keyboard focus (for example, bymouse-clicking on a designated button in that window), the wedge serveruses the COM protocol to pass audio/video key events to the applicationspecified by the user, regardless of which application window in theforeground has the text keyboard focus.

There are several user interfaces that can allow the user to specifywhich window is to retain the audio/video keyboard focus. One method isfor each event client program to present the user with a mouse-clickablebutton that, when activated, causes the event client 72, 73 to notifythe wedge server 71 program that audio/video key events should bedelivered only to that event client program. However, this method hasthe drawback that a malicious event client program may notify the wedgeserver that it is to hold the audio/video keyboard focus even if theuser has not requested this behavior.

An alternative user interface eliminates this problem by having the userinteract with the wedge server 71 program window in order to specifywhich program is to receive the audio/video keyboard focus. Each eventclient program, when registering its existence with the wedge server,specifies a word or phrase that identifies the program to the user(preferably, the word or phrase in the program's main window title bar).The user then observes the wedge server 71 window which displays a listof programs that may hold the audio/video keyboard focus. The userindicates which one of these programs is to hold the audio/videokeyboard focus by moving the mouse over the corresponding identifyingphrase and clicking the mouse.

Regardless of the user interface presented to the user, when the userspecifies a window to hold the audio/video keyboard focus, any otherwindow that presently has the audio/video keyboard focus immediatelyloses the focus in favor of the more recently specified window. Also,there is a user interface means for the user to specify that theaudio/video focus should return to the default mode where theaudio/video keyboard focus is the same as the text keyboard focus. Thiscan be accomplished several ways, such as including a mouse-clickablebutton in the wedge server 71 window that causes the server to enter thedefault mode. Or the same user interface that is used to specify theselection of the audio/video focus may be used to turn off the focus tothat window. That is, if the user selects a window for the audio/videokeyboard focus that already has the focus, this is interpreted as adesire to turn the focus off of that window, and enter the default mode.

Thus, according to the above methods, the user may turn the audio/videokeyboard focus on or off for any audio/video key event client program,and only one such application may hold the audio/video keyboard focus atany time.

4. Multiple Keyboard Focuses Built into an Improved Operating System

It is common for text keyboard focus and key event processing to bebuilt into the operating system so that another program running on theoperating system cannot maliciously change text keyboard focus from theuser's specification. In the embodiment described above, the wedgeserver and event client programs are not built into the operatingsystem, thus they are subject to interference from other programs.Furthermore, the above embodiment requires significant work on the partof playout program developers in order to ensure that their programscomply with the wedge server event delivery architecture. Also, theabove embodiment requires special registration methods to allow thewedge server program to gain a reference to each application that mayreceive audio/video key events. In contrast, the operating system isresponsible for loading computer programs, thus it already has a meansof finding and passing information to all executing programs.

Thus, in a preferred embodiment, the audio/video keyboard focus isincorporated directly into the operating system, so that neither theuser, nor another program may inadvertently or intentionally misdirectthe delivery of audio/video key events or the assignment of theaudio/video keyboard focus.

In this preferred embodiment, audio/video key events are delivered in asimilar manner as regular text keyboard events. This means that programsrunning on the operating system require minimal additional programmingeffort in order to interpret audio/video key events, since they can usethe same software architecture as regular key event processing.Audio/video keyboard focus selection is built into the operating systemto prevent malicious programs from changing the audio/video focus to awindow that was not specified by the user.

In contrast to existing operating systems, the audio/video keyboardfocus behaves in the manner described by the previous embodiment. Thatis, by default the audio/video keyboard focus follows the text keyboardfocus. However, the user may set the audio/video keyboard focus toanother computer program. For example, in Microsoft Windows the textkeyboard focus can be set by moving the mouse over a program window andclicking the left mouse button. If the right button is clicked, a“pop-up” window displays various actions to perform on that window (suchas minimize, close, or maximize). One of such options could be to holdthe audio/video keyboard focus. Alternatively, just as most programsinclude mouse clickable buttons in their title bars in order tominimize, maximize or close the window, an additional button may beincluded to turn on or off the audio/video focus to that window. Or aspecial key or combination of keys (such as Ctrl-Alt-Insert) may bepressed while the desired window is in the foreground, which causes theforeground window to hold the audio/video focus if another window issubsequently assigned the text keyboard focus. Alternatively, a specialkey or combination of keys may be reserved to cycle through allexecuting computer programs that can receive the audio/video focus,where each time the special key is pressed, the next executing programis assigned the audio/video focus (perhaps indicated visually by adifferent color window border). This particular approach is mostusefully applied using the keys on a hand-held keypad so that theaudio/video focus can be selected from a distance without using themouse or text keyboard.

Distinction Between Local Key Presses and Remote Key Presses

Contemporary computer systems may include either a keyboard designed forshort-range (local) usage (such as a traditional text keyboard as inFIG. 3) or a keyboard designed for long-range (remote) usage (such as ahand-held remote as in FIG. 2) or both such keyboards. Systems with suchdual keyboard capabilities make no attempt to indicate to audio/videoplayout programs whether a key press occurred on a short-range keyboardversus a long-range keyboard.

This is valuable information to a playout program, because a differentmode of operation may be desirable depending on which keyboard is beingused. For example, a video playout program may automatically enter afull-screen mode if it detects that the user has pressed a key on along-range (remote) keyboard. Or an audio playout program maytemporarily display a large on-screen indicator (more easily viewablefrom a distance) of the audio output volume if a volume adjustment keyis pressed on a long-range keyboard.

Rather than requiring the user to notify a program to increase itsdisplay size if a long-range keyboard is being used (as would be done oncommon systems), it is better if the program is notified with eachkey-press whether the key originated from a short-range or long-rangekeyboard. This is accomplished by adding just one bit to each key eventmessage. A “one” bit indicates that the key originated from a long-rangekeyboard, and a “zero” bit indicates that the key originated from ashort-range keyboard. Alternatively, an entirely new set of key codesmay be used for each key on a long-range keyboard, if those keys existin duplicate on short-range keyboards.

In this manner, the user may easily switch between short-range keyboarduse and long-range keyboard use, and the playout program is notified ofthis change by each new key press.

There are several ways to improve a computer program's user interface ifthe program is able to detect the difference between local versus remotekey presses. For example, the user may run a video playout program thatdisplays a video clip in a window. When a remote key press is detected,the program enlarges the video to occupy the entire screen to makelong-range viewing easier. This is done until a local key press isdetected, and the playout program returns to its original size andlocation on the screen. Or a user may run an audio playout program thatnormally displays its volume setting in a small graphical indicator.When the user presses a remote button to change the volume, the programdisplays a large, full-screen volume indicator that can be seen fromlarge distances. This indicator is displayed for a programmable amountof time (for example 0.5 to 5 seconds) in order to allow the user to seethe volume setting. After this pre-programmed time elapses, the programno longer displays the large volume indicator, so that other on-screengraphics may be viewed more easily. This same method can be employedwhen the user presses buttons to fast-forward or rewind an audio/videoclip. In this case, the on-screen indicator is a large horizontal“progress bar” whose degree of opaqueness indicates the current point ofplayout in the clip. That is, if the song is halfway through playout,the progress bar is half opaque, and the other half is the screen'sbackground color.

The above described embodiments of the invention are only examples. Manyother embodiments are possible. The scope of the invention should not belimited to the above embodiments but should only be limited by thefollowing claims:

1-18. (canceled)
 19. A method comprising: providing code for download toand execution by a processor-based system having an operating system;via the code, and responsive to detection of information representinghuman selection of a key made via a user interface device causing theprocessor-based system to compare the received information with one ormore event values associated with a media player program, each eventvalue representing selection of a single audio or visual control commandto be executed by the media player program; and via the code, causingthe processor-based system to detect when there is correspondencebetween the received information and the one or more event values, andto responsively direct a control command corresponding to the one ormore event values to the media player program to execute thecorresponding control command.
 20. The method of claim 19, wherein theprocessor-based system is to execute a browser program, and wherein: thecausing of the processor-based system to detect when there iscorrespondence comprises providing code for execution in associationwith the browser program, the code when executed to cause theprocessor-based system to detect when there is correspondence betweenthe received information and the one or more event values, and toresponsively direct a control command corresponding to the one or moreevent values to the media player program to execute the correspondingcontrol command; and the method further comprises causing of theprocessor-based system to detect when there is no correspondence betweenthe received information and the one or more event values, and toresponsively direct the received information to at least one of theoperating system or a non-media player program executing on the browseror the operating system.
 21. The method of claim 20, wherein: thecausing of the processor-based system to compare comprises causing theprocessor-based system to compare a value of the key corresponding tothe human selection with a list representing one or more predeterminedkeys; the causing of the processor-based system to responsively directthe control command corresponding to the one or more event values to themedia player program comprises causing the processor-based system toconvert the value of the key to a predetermined audio or visual controlcommand associated with the value of the key, and to provide thepredetermined audio or visual control command to the media playerprogram; and the causing of the processor-based system to responsivelydirect the received information to the at least one of the operatingsystem or non-media player program executing on the operating systemcomprises withholding the received information from the media playerprogram.
 22. The method of claim 21, the causing of the processor-basedsystem to responsively direct the control command corresponding to theone or more event values to the media player program compriseswithholding the control command from provision to the at least one ofthe operating system or non-media player program.
 23. The method ofclaim 20, wherein the causing of the processor-based system toresponsively direct the received information to the at least one of theoperating system or non-media player program comprises causing theprocessor-based system to provide an ASCII character to the browserprogram corresponding to the value of the key.
 24. The method of claim23, wherein the browser program comprises two or more windows that areto be displayed by the processor-based system, and wherein the causingof the processor-based system to provide the ASCII character to thebrowser program comprises causing the processor-based system to:identify a first one of the two or more windows which is to have focus;and provide the ASCII character to the first one, and withhold the ASCIIcharacter from at least one second window of the two or more windows.25. The method of claim 19, wherein: the method further comprisescausing the processor based system to convert the informationrepresenting the human selection of the key made into an ASCIIcharacter; and the causing of the processor-based system to comparecomprises causing the processor-based system to compare the ASCIIcharacter with the one or more event values.
 26. The method of claim 19,wherein the value of the key corresponds to a media control key of ahandheld remote control.
 27. The method of claim 19, wherein theproviding of the code comprises providing script which is executable bya browser program that is to be run on the operating system.
 28. Themethod of claim 19, wherein the media player program is a web-embeddedmedia player program and wherein the causing of the processor-basedsystem to responsively direct the control command corresponding to theone or more event values to the media player program to execute thecorresponding control command comprises causing the processor-basedsystem to call a function written according to the document object model(DOM), in order to invoke the corresponding control command.
 29. Anapparatus comprising instructions stored on non-transitorymachine-readable media, the instructions adapted for download to andexecution by a processor-based system having an operating system, theinstructions when executed to: cause, responsive to detection ofinformation representing human selection of a key made via a userinterface device, the processor-based system to compare the receivedinformation with one or more event values associated with the mediaplayer program, each event value representing selection of a singleaudio or visual control command to be executed by the media playerprogram; and cause the processor-based system to detect when there iscorrespondence between the received information and the one or moreevent values, and to responsively direct a control command correspondingto the one or more event values to the media player program to executethe corresponding control command.
 30. The apparatus of claim 29,wherein the processor-based system is to execute a browser program, andwherein: the instructions further comprise code to be executed inassociation with the browser program, the code when executed to causethe processor-based system to detect when there is correspondencebetween the received information and the one or more event values, andto responsively direct a control command corresponding to the one ormore event values to the media player program to execute thecorresponding control command; and the instructions, when executed, arefurther to cause of the processor-based system to detect when there isno correspondence between the received information and the one or moreevent values, and to responsively direct the received information to atleast one of the operating system or a non-media player programexecuting on the browser or operating system.
 31. The apparatus of claim30, wherein the instructions, when executed, are further to cause theprocessor-based system to: compare a value of the key corresponding tothe human selection with a list representing one or more predeterminedkeys; responsive to detection of correspondence between the receivedinformation and the one or more event values, convert the value of thekey to a predetermined audio or visual control command associated withthe value of the key, and to provide the predetermined audio or visualcontrol command to the media player program; and responsive to detectionof no correspondence between the received information and the one ormore event values, withhold the received information from the mediaplayer program.
 32. The apparatus of claim 31, wherein the instructions,when executed, are further to cause the processor-based system to,responsive to detection of correspondence between the receivedinformation and the one or more event values, withhold the controlcommand from provision to the at least one of the operating system ornon-media player program.
 33. The apparatus of claim 30, wherein theinstructions, when executed, are further to cause the processor-basedsystem to provide the received information to the browser program in theform of an ASCII character corresponding to the value of the key. 34.The apparatus of claim 33, wherein the browser program comprises two ormore windows that are to be displayed by the processor-based system, andwherein the instructions, when executed, are further to cause theprocessor-based system to: identify a first one of the one or morewindows which is to have focus; and provide the ASCII character to thefirst one, and withhold the ASCII character from at least one secondwindow of the two or more windows.
 35. The apparatus of claim 29,wherein the instructions, when executed, are further to cause theprocessor-based system to: cause the processor-based system to convertthe value of the key into an ASCII character; and compare the ASCIIcharacter with the one or more event values.
 36. The apparatus of claim29, wherein the value of the key corresponds to a media control key of ahandheld remote control.
 37. The apparatus of claim 29, wherein theinstructions comprise script which is executable by a browser programthat is to be run on the operating system.
 38. The apparatus of claim29, wherein the media player program is a web-embedded media playerprogram and wherein the instructions, when executed, are further tocause the processor-based system to call a function written according tothe document object model (DOM), in order to invoke the correspondingcontrol command.